Andrew Ng unveils vLLM course with Red Hat teaching KV cache memory management techniques in transformer model serving and history and technical architecture of vLLM LLM inference engine for 70B models.

Tether AI upgrades QVAC SDK with TurboQuant, delivering data center-sized memory for local AI inference on everyday devices.

Tether AI releases TurboQuant KV-Cache Quantization in QVAC SDK 0.12.0, cutting memory use 5x near-lossless for stronger local AI on edge devices.